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We study the large time fluctuations of entropy production in Markov processes. In particular, 

I— I ■ we consider the efltect of a coarse-graining procedure which decimates fast states with respect to a 

' given time threshold. Our results provide strong evidence that entropy production is not directly 

O ^ affected by this decimation, provided that it does not entirely remove loops carrying a net probability 

^ . current. After the study of some examples of random walks on simple graphs, we apply our analysis 

' to a network model for the kinesin cycle, which is an important biomolecular motor. A tentative 

^ I ^ \ general theory of these facts, based on Schnakenberg's network theory, is proposed. 
■ 
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S ■ I. INTRODUCTION 

^ : 

\ The coarse-graining procedure is a fundamental ingredient of the statistical description of physical systems 
^ ' By coarse-graining we mean a procedure which reduces the number of observables to simplify the physical description. 
I I For instance, it is used to describe the behaviour of the physically relevant quantities, or slow variables, which depends 
on the coupling among all variables characterizing the system of interest, including the so-called fast variables. The 
' archetype of such a procedure is the treatment of Brownian colloidal particles, immersed in a fluid, in terms of the 
^ , Langevin equation. In this sense any model meant to represent a real phenomenon may be thought of as a coarse- 
grained, i.e. reduced, description. The purpose of a model is, indeed, to advance our understanding of the object 
ly-j under investigation, by highlighting its interesting features and discarding the irrelevant ones. In turn, the roles of 
^ relevant and irrelevant characteristics depend on the purpose of the analysis to be performed. Furthermore, it isn't 
^ I always obvious which quantities should be listed as interesting, and which ones should be neglected, especially if a 
. new problem is to be tackled [iHll- Therefore, it is critical to understand how specific physical observables depend on 
' the coarse-graining procedure. 
^-H , Examples of coarse-grained descriptions at different resolution levels include the steps meant to connect the mi- 
' croscopic descriptions of systems of physical interest to the macroscopic ones, for instance the passage from the 
. ^ ' deterministic F-space description (positions and momenta of the N particles) to the stochastic /i-space description 
(position and momentum of one particle), up to macroscopic descriptions such as hydrodynamics, Fourier law, Navier- 
^ Stokes equations, etc. 

Other methods use the coarse-graining procedure in order to reduce the number of variables, e.g. by a decimation 
method which suppresses the fast variables, or perform a spatial coarse-graining, as in the renormalization group 
approach. In these methods the coarse-graining is parametrized by some threshold, here denoted as coarse-graining 
level (CGL). This paper is devoted to the investigation of the impact of variations of CGL on the entropy production 
of non-equilibrium systems. 

In the last decades, the introduction of the so-called Fluctuation Relations (FR) for deterministic dynamics, by 
Evans, Cohen, Morriss, Gallavotti, Jarzynski and other authors brought about important developements in the 
physics of far from equilibrium systems @-l8|. In the specific context of Markov processes, here discussed, Lebowitz 
and Spohn showed that the "entropy production" per unit time, measured on a time-interval t, Wt say, is described 
by a large deviation theory whose Cramer function, C, enjoys the following symmetry property: 



C{Wt) - C{-Wt) = -Wt 





FIG. 1: A simple example of decimation which results in vanishing entropy production. 



This relation is the stochastic counterpart of the deterministic steady state FR, and we call it Lebowitz-Spohn FR, 
or simply FR. 

We remark that the FR does not provide any specific information about the shape of C. Therefore, a coarse- 
graining procedure which preserves the Markovian character of the model should preserve the validity of the FR as 
well, although it may change the shape of the Cramer function. As a matter of fact, Rahav and Jarzynski fl^ argue 
that the validity of the FR is little affected by the coarse-graining procedure, even in nontrivial cases, such as those 
in which the decimation (or blocking) of variables results in the loss of the Markovian property. 

In the present paper, at variance with Ref.[l0|, we do not address the question of validity of the FR (which is always 
satisfied by our models), but focus our attention on the effects of the decimation procedures on the behaviour of the 
Cramer function. 

Understanding how C changes under variations of the CGL is relevant, e.g. to interpret experimental results, since 
they are always obtained at finite resolution (for instance in frequency, [^). Likewise, any model describing a real 
system is necessarily affected by some degree of approximation or of idealism. For instance, the entropy production 
defined by Lebowitz and Spohn appears to be a rather abstract quantity, depending on the direct and inverse trajec- 
tories in the state space, as well as on their probabilities in the stationary state, cf. Section II. Furthermore, such a 
quantity cannot be measured in a direct way. Therefore, it needs to be connected to directly measurable quantities, 
for its properties to be assessed. 

Naively, one may expect the entropy production computed through a model which encompasses lots of details of the 
system of interest to be higher than that computed through less detailed models. Consider for example the Markov 
chain depicted in Fig. [1^, with transition probabilities 2^-1 and 3^-4 much larger than the remaining ones. One 
may impose that a net current flows from 3 to 1 and from 2 to 4, by choosing transition probabilities Ps^i Pi-^z 
and P2-S-4 ^ and by tuning the other parameters so that all states have the same stationary probability. In this 

system, detailed balance does not hold, hence the mean entropy production is positive. However, the mean entropy 
production vanishes if the fast states 2 and 3 are decimated, and the Markov chain is reduced to the one represented 
in Fig. [T|d, where A corresponds to the old state 1 and B to the old state 4. Indeed, detailed balance, which implies 
a vanishing entropy production, holds in any Markov chain with two states. 

In the following we consider Markovian systems described by a Master equation: 



(1) 



where Wi^n is the transition rate from state I to state n, with l,n £ [1,N], N being the number of possible states of 
the process, Pn{t) is the probability to stay in the state n at time t, and 



(2) 



We adopt the coarse graining procedure introduced in Ref. and described in Appendix \^ which amounts to a 
decimation of all states whose characteristic times t„ = 1/W^ are smaller than a given At > 0. The resulting master 
equation for the surviving states may be written as 



(3) 



with transition rates Wi^j as prescribed by [III, ^ [1, N < N, and Wj < l/Ar for aU j e [1, N] 
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Varying Ar the number of the slow states and the shape of the Cramer function CAr(VVt) may in principle change. 

In Section II, we present some numerical results for differently decimated Markov processes. In section III, we 
discuss the possibility of constructing a general theory, not yet available, of the decimation effects. In Section IV, we 
draw some conclusions and discuss open problems. Appendix A illustrates the decimation procedure; Appendix B 
reports analytical results about the effect of decimation on the current in a single loop; Appendix C recalls the graph 
analysis of currents in a Markov system, based on Schnakenberg's theory; Appendix D describes in detail the kinesine 
model mentioned at the end of section II; Appendix E lists the main symbols used in the text. 



II. SOME NUMERICAL RESULTS 

A. Entropy production on a trajectory 

While the concept of entropy production, or energy dissipation, dates a long time back [l^ . only recently have 
the fluctuations of entropy production attracted particular interest, thanks to important theoretical and numerical 
results, supported by some experimental evidence. For a trajectory of duration t of a continuous time Markov process, 
in which m transitions ujq — > uji...ujm-i uJm are observed, uji being the i-th visited state, the following definition of 
entropy production has been given by Lebowitz and Spohn [o^ : 

It can be shown that the times ti at which the transitions occur affect the numerical value of Wf only with corrections 
of order 0{l/t), negligible in the i — t- oo limit. In the present paper, we consider this entropy production, introduced 
by Lebowitz and Spohn. Clearly, this quantity does not need to represent any real thermodynamic observable, since 
it can be defined independently of the physical relevance of the Markov process at hand. Nevertheless, as commonly 
done in the literature, we will refer to it merely as to "entropy production" . 

Even if in this paper we consider the case of continuous time, in the discrete time case (Markov chains) one can 
use the same definition by replacing Wi^j with the probability of a transition in a time step 5t, Pi^j- The 
relation between continuous and discrete time quantities is incorporated in the equalities Pi^j = Wi^jSt for i ^ j, 
and P,^i = 1 - Wj^St. 

The connections of Wt with other definitions of entropy production rate are discussed in Appendix [C] Here, it 
suffices to recall that (Wt) = in the steady state, if the invariant probability p™^ of the process satisfies the detailed 
balance condition 

VVu^ = fi£_. (5) 

The system is in equilibrium if eq. ^ holds. If detailed balance does not hold, one has (Wt) > 0. Let C be the 
Cramer function of the probability density function (pdf ) / of Wt , in the steady state, i.e. let C be defined by 

C(Wt) = - lim ilog[/(Wt)]. (6) 

t->-oo t 

In numerical calculations, the Cramer function C must be approximated by its finite time counterparts, C(Wt) ~ 
— log[/(Wt)]/i. Therefore, in our calculations, we have chosen times t large enough that further growths of the 
averaging times practically do not affect our results. 

Lebowitz and Spohn have shown that the condition 

C{Wt) - C(-Wt) = -Wt, (7) 

is better and better approximated as the time t grows [9| . The t ~^ oo limit of relation ([7]) is known as a Steady State 
Fluctuation Relation (SSFR). It does not provide the shape of C, but only a symmetry property of C. Remarkably, 
C is system-dependent [l^, while ([7]) holds quite in general. 

In the following, we address the question of the dependence of C on the CGL which, in the decimation procedure 
of is parametrized by the threshold time At. The protocol of Ref . . eliminates all states i with average exit 
time Ti < At, and requires the surviving states to have re-normalized transition rates Wui-^ui' ■ Denote by /At and by 
CAt the pdf of the entropy production and its Cramer function, for the decimated process with threshold time At. 

The present investigation suggests the conjecture that the entropy production does not depend sensibly on the 
precise properties of fast and slow variables of the Markov process: it only depends on the currents fiowing in the 
system. 
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B. Results on Id and 2d regular lattices 



Let us begin focusing on continuous time random walks on simple topologies, i.e. on regular lattices with periodic 
boundary conditions, with random transition rates restricted to nearest neighbours. To simplify the procedure. 



we require every state n, n = 1, 



{r 



(1) 



r(M) 



} such that 



-1) 



< T 



to have characteristic (exit) time t„ = 1/ X^z^^n belonging to a set 

^ -^("+1), This condition corresponds to a separation of time-scales which 



represents a mild requirement for the decimation protocol of [11| to apply. Transition rates may then be chosen 
to have, or not to have, a preferential direction, in order to allow, or to prevent, a positive entropy production. 
For instance, entropy production can be positive in Id lattices, only if some rates obey Wi^i+i/Wi^i^x / 1 (cf. 
Appendix [Cl for a more precise condition, based on the notion of affinities). 

Simulations for regular lattices show a striking robustness of the entropy production Cramer function with respect 
to decimation. The numerically computed Cramer function — log /At (Wt)/t for Id chains is plotted in Figure [2] 
for different CGL. The figure shows that decimating 90% of the system, i.e. leaving only the slowest states, the 
fluctuations of entropy production remain substantially the same. The result does not seem to depend on the details 
of the transition rates, but only on the separation of time-scales. 

In the following sections, we show that the entropy production is not directly related to the properties of fast and 
slow states per se either, while it seems reasonable to conjecture that it is closely related to the currents flowing in 
the system. These are global quantities, rather than local ones, which depend on the topology and on the interplay 
among all transition rates, an idea that may be understood in simple terms, as follows. In the case of a random walk 
on a ring (a 1-dimensional lattice with periodic boundary conditions), one may write 



Wback 



X n 



(8) 



where W/o™ = Wi^2W2-^3---Wn~i^nWn^i, Wback = Wi^nWn^n~i---W3^2W2^i, m is an integer and 11 « 0{1) 
is a correction term. In presence of a mean current J > 0, one has m = [G{t)t\ where G{t) is the current computed 
in the time window (0,<) (i.e. limt^oc {G{t)) = J) and [a] indicates the integer part of a. This leads to the relation 



Wt « G{t) lof 



Wfo 



+ 0{l/t). 



(9) 



Wback 

Since the decimation protocol eliminates fast states and modifies transition rates in order to leave basically unaltered 
the currents connecting the surviving states, the fluctuations of entropy production should not be sensibly affected 
by the decimation procedure. Observe, however, that this current conservation is only approximate, not exact. The 
modification of the current produced by the decimation can indeed be estimated analytically in simple cases, as in 
systems whose states form a single loop. In Appendix [B] we argue that the correction should be generally small and 
related to the ratio of the times spent in the fast states and in the slow ones. 




Q-Q Originul system 
B-B At=0.05 (70% sites decimated| 
y -- At=0.5 (90% sites decimated) 



0.8 



FIG. 2: Numerically computed Cramer function for entropy production distribution in ID continuous time random walks with 
p.b.c. In each plot a comparison among different coarse graining levels is shown. In both panels, each state of the non-decimated 
system may take one out of three possible average exit times: 1 (10% of states), 0.1 (20% of states) and 0.01 (70% of states). 
In the left panel, the probability of jumping to the right is 0.4 (60% of states) or 0.6 (40% of states). In the right panel, the 
probability of jumping to the right is 0.4. In the left frame we have A'' = 100 and t = 2 ■ 10"^. In the right frame we have 
A'' — 300 and t = 10"'. The numerical computation has been performed with the Gillespie algorithm p^ . where the actual 
probability of a transition is the product of the transition rate by the characteristic time. 



Similar results are reported in Figure|3]for 2D regular square lattices, where jumps occur among nearest neighbours: 
even with this topology, the fiuctuations of entropy production appear not to be affected by the CGL, although the 
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result is not as robust as in the ID case. Indeed, a substantial change in entropy production can be observed if the 
system contains a very large number of fast states to be decimated. Nevertheless, it still is interesting to realize that 
the Cramer function has not changed substantially, even after 50% of the original system has been decimated. Note 
that the square lattice topology is drastically altered by decimation: states which have not been decimated remain 
connected by chains of transitions, but the system is not planar nor a regular lattice anymore. Currents in this case 
may still be defined within a more general graph theory [1^ , such as that discussed in Appendix O 




FIG. 3: Approximated Cramer function for entropy production distribution in continuous time random walks on a 2D squared 
lattice (nearest neighbours) with p.b.c. In each plot a comparison among different coarse graining levels is shown. In both 
cases the probabihties of jumping to one of the four nearest neighbours are biased to give a net current in one direction. The 
left and the right panels differ by the values of the exit times. Left: the states have exit times 1 (50% of states), 0.1 (20% 
of states) and 0.01 (30% of states). Right: the states have exit times 1 (70% of states), 0.1 (20% of states) and 0.01 (10% of 
states). In both cases iV = 100 and t = 1000 



C. Results on graphs with fast and slow loops 

Guided by the conjecture that the fundamental ingredient for entropy production is the current flowing in a circuit, 
we construct Markov processes composed of independent loops joined by a single interchange state. The general 
structure of this graph is illustrated in Fig. HI The main slow loop is decorated by fast loops (first level), which are 
on their turn decorated by faster loops (second level), etc. After decimation, one may encounter different situations: 

1. the new and the old structures have the same topology, i.e. only pieces of loops have been suppressed but the 
number and position of loops is the same; 

2. all loops of the faster (outer) level are suppressed; 

3. all loops of the two fastest levels are suppressed; 

4. and so on; 

Loops at the same level have similar properties and, in particular, are chosen to have, or not to have, a positive 
entropy production, i.e. to have or not to have a preferential direction in their transition rates. 

The computed Cramer function for the entropy production of the case of Figure |4] is reported in Figures [5] and |6l 
In Figure [5] the states of the fast loop have slightly different characteristic times, allowing a progressive decimation 
of the fast loop. At a decimation threshold such that the fast loop is still alive, even if made of only three states 
(blue curve), we are in situation 1 and the Cramer function of the entropy production is very close to that of the 
non-decimated system (black curve). A further increase of the decimation threshold makes the fast loop disappear: it 
remains with only two states and the system falls in situation 2, where the Cramer function of the entropy production 
has a sudden macroscopic change (red curve). The inset of Figure [5] shows (Wt) as a function of the decimated 
percentage of the fast loop: neglecting a very weak growth, (Wt) appears practically constant, until the fast loop is 
not reduced to a 2-states branch. If the main loop is configured to have a non-zero current, this is what remains at 
that point, otherwise the fiuctuations of Wt are reduced to a very narrow and symmetric peak around zero. 

Figure [S] shows other cases with three levels of loops. The Cramer function of the entropy production always changes 
when a level of current-carrying loops is entirely removed. 



FIG. 4: Sketch of a graph made of three levels of nested loops: in this example, the main level loop has no preferential direction, 
while the second and third levels are made of smaller loops with faster states and preferential directions. 




FIG. 5: Approximate Cramer function for entropy production distribution in continuous time random walks on a graph similar 
to that of Figure |4l with two hierarchical levels. A comparison among different coarse graining levels is shown. The main 
loop is made of 100 states with average exit time 1 and preferential direction given by balanced (left) or unbalanced (right) 
transition rates. In the case of unbalanced rates, they are 0.6 toward left and 0.4 toward right. The second level loops have 30 
states and a bias in the transition probabilities (0.8 vs. 0.2) chosen to give a preferential direction, while their characteristic 
times range from 0.1 to 0.7. In all simulations t — 1000. 



D. A model from molecular biology: coarse graining of the Kinesin's network 



The examples in the previous sections suggest that the entropy production is weakly affected by the coarse graining 
procedure, apart from the cases in which loops contributing significantly to the entropy are destroyed and the entropy 
production undergoes an abrupt decrease. A natural question is whether this phenomenology is a peculiarity of the 
model introduced here, or similar behaviors pertain to other realistic models of non-equilibrium systems. Biochemical 
reactions are often characterized by non-equilibrium processes acting over different timescales and thus afford an ideal 
benchmark for the ideas proposed here. In this subsection, we study the effect of coarse graining on a recent network 
model of the kinesin motor cycle [l^ . 

Kinesins are a common category of motor proteins 17[ that are used for transport on microtubules in eucaryotic cells. 
Like many other non-equilibrium reactions inside cells, kinesin is powered by ATP. A number of experiments during 
the last decades elucidated many structural and dynamical details of this systems. In particular, it is now understood 
that kinesins are made of two identical heads, that walk on microtubules with a "hand-over-hand" mechanism [is} , 
alternating their position at the front. 

The model proposed in [l^ describes both the ATP-driven chemical reactions and the mechanical step in which 
the two heads swap. The multiple cycle structure of the reaction is given by this chemomechanical nature and by 
considering the fact that ATP may be burned by both heads. The scheme of the reaction and the possible transitions 
are illustrated in Fig. ([7]). Each of the two heads may be in three different configurations: free, bound to ATP and 
bound to ADP, resulting in 3 x 3 = 9 possible states. However, the motor is believed to work "out of phase", i.e. 
states in which the two heads are in the same configuration are unlikely to be observed. This reduces the model 
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FIG. 6: Approximate Cramer function for entropy production distribution in continuous time random walks on the topologies 
of Figure m with three hierarchical levels of nested loops and t = 10^. In each plot a comparison among different coarse grained 
levels is shown. The main loop is made of 100 states with average exit time 1 and no preferred direction. The second level loops 
(each with 10 states) have average exit time 0.1 and a bias in the transition probability chosen to give a preferential direction. 
The third level loops (each with 5 states) have average exit time 0.01. The difference between the left and right frames is in 
the transition rates of this third level. Left: third level loops have a preferential direction. Right: third level loops without 
preferential direction. 



P 




FIG. 7: Scheme of the transition network of the kinesin model [T^. a) Reaction network. The states, numbered from 1 to 
6, are characterized by the two heads bound to ATP (A), ADP (D) or free. The molecules bound (or released) during the 
chemical transitions are shown as connected to the arrows. The dashed arrow represents the mechanical transition, in which 
kinesin makes its step on the microtubule, b) Kinesin network after coarse graining the fastest states, first state 2, then state 
5. Dot-dashed arrows represent the new transitions appearing as a consequence of the coarse graining procedure. 

to the 6 states represented in the scheme of Fig. Of all the possible transition among the states, only those 

which are consistent with experimental observations are considered in the model and shown in the diagram. Clearly, 
the assumptions above (in particular that of considering only 6 states) already imply some level of coarse graining 
with respect to the complete problem. However, the effect of these assumptions on the entropy production is hard to 
determine, since it would be difficult to construct a more detailed model, from the available experimental results. We 
then take the model of Fig. ^ as our starting point, and study the effect of decimating the states of the system. 

The transition rates of the model depend on the ADP, ATP and P concentration and on the load force F of the 
molecular motor. Moreover, the parameters determining these rates have a slight dependence on the kind of the 
experiment one wants to reproduce, since different experiments work in different conditions and may use different 
kinds of proteins in the kinesin family. We determined the rates by choosing the parameters fitting the experiment 
of Rcf. 19] and assumed fixed concentrations of [ADP] = [ATP] = [P] — IfiM (micromoles) for simplicity. We then 
consider two different cases: one without work load and one with a work force equal to F ^ 5pN (piconewton) . 
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Details on the derivation of the rates and numerical values are given in Appendix (ID|) . 

In both cases (with and without load), state 2 is the fastest and state 5 is the second fastest. We compare then the 
entropy production of the complete model, of the model in which state 2 has been adiabatically eliminated and of the 
model in which both 2 and 5 have been eliminated. The pdf of finite-time-averaged entropy production Wt^^^ /tmax, 
obtained from 10000 realizations of trajectories of length tmax is shown in Figure |8l without load in the left panel, 
with load in the right panel. Decimation of state 2 leaves the pdf unchanged. In the case without load, further 
decimation of state 5 changes abruptly the pdf to a close-to-zero peaked pdf. 




FIG. 8: Pdf's of the entropy production per unit of time. Black curve is for the original model with 6 states. Red is after 
decimation of state 2. Green is after decimation of states 2 and 5. Left panel: the case without load. Right panel: the case 
with load. 



This model reproduces the same scenario of the "fast loop-slow loop" model of the previous section. The first coarse 
graining strongly alters the structure of the network, but its effect on the entropy production and its fluctuations 
is barely noticeable. Conversely, decimating one more state drastically reduces the entropy production. Notice also 
that the most irreversible transition in the original model is the "mechanical" transition between state 2 and 5, since 
1^25/ W^52 = 3 10"^ for the load-free case and w 57 for the loaded case. In the sense specified in the next section, the 
information about the irreversibility of this transition is lost, when the level of coarse graining is too large. 



III. TENTATIVE THEORY 



Consider a continuous time Markov process: each state n can be seen as a vertex of a graph, and transitions n ^ n' 
with a positive rate correspond to edges (also called links) between n and n' . As in [9,], we assume that the transition 
n ^ n' has a positive rate whenever the inverse transition n' ^ n does. As illustrated in Section 2.B, the entropy 
production of random walks on ID rings is closely related to the current flowing in the ring. In this Section, we 
attempt to generalize this observation to generic graphs [2^. The main tool for this purpose is a decomposition 
in fundamental cycles, which is illustrated in Appendix [C] In the example of Figure |4l the fundamental cycles are 
nothing but the loops. 

Let us introduce a different functional Qt : 

Qt = J2A{Cc.)G^{t) (10) 

a 

which depends only on a few "structural properties" of the process: the fundamental cycles Ca (i.e. a property of the 
graph), their ajfinities A(Ca) and their fluctuating currents Ga, averaged over a time interval (0,t), which depend 
also on the transition rates. 

In various numerical simulations, we have verified that the fluctuations of Qt and those of Wt are practi cally 
indistinguishable, at large times: cf. FigurelHlfor two examples. It is also known that (Qt) = {Wt) for large times [2l| . 

To obtain a complete theory, it remains to show that the "structural properties" are not affected by the decimation; 
this task can be subdivided in three steps: 

1. examine the fate of fundamental cycles after decimation of one fast state; there are three possibilities: i. cycles 
may be destroyed, ii. transformed into different ones, iii. new cycles may be created; 
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W,, Q_st W,, Q, 

FIG. 9: Equivalence of the Cramer function for Wt and Qt aXt — 10'*. Left: random walk on a 2d lattice with same parameters 
as in Figure |3] (right). Right: random walk on a nested loop graph with same parameters as that in Figure |6] (right). 

2. derive the corresponding variation of affinities; 

3. obtain the values of currents after decimation. 

Concerning task 2, it is easy to realize that affinities do not change in transformed cycles, while new cycles have zero 
affinity, so they do not contribute to the entropy production. Disappearing cycles pose, instead, a difficult question: 
numerical simulations show that they are usually small and that the affinity lost with their removal is equally small. 
At the moment, however, we do not have an analytical estimate of this quantity. Task 3 is a hard problem too: 
the stationary value of currents must satisfy many coupled Kirchhoff equations and depends on the properties of the 
whole graph. Numerical simulations suggest that average currents are not drastically influenced by our decimation 
procedure. One rough explanation of this fact can be given for a system S with small entropy production, obtained 
from a perturbation of an equilibrium system TP. Indeed, one may assume a linear relation between the affinities 
A and the average currents Jp of E, of the form A{Ca) = X]/3 La^Jp, with coefficients Lq/j determined only by the 
properties of E". If the decimation procedure, which replaces S with a new system E', leaves substantially unaltered 
the invariant probability of the surviving states, it is reasonable to assume that the decimated system E' is another 
small perturbation of E°. Then, the linear relation between affinities and currents of E' retains the same coefficients 
Lq.^, leading to the conclusion that the currents are conserved under decimation, if affinities are. 

We now discuss the consequences of decimation on cycles and their affinities. A maximal (also called "spanning") 
tree T is found on the original graph G. This tree includes all N vertices (states) and only a part {N — 1) of the 
original number E of edges. All pairs of vertices are connected by a unique path on this tree. All edges left out 
from the tree (a number v ^ E — [N — 1)) are called "chords". A chord connecting vertices i and j, attached to 
the unique path connecting i and j along the tree, is a closed loop. All loops generated in this way constitute the 
set of fundamental loops, which become "cycles" when orientation is taken into account. These fundamental cycles 
determine the statistics of Qt and therefore of Wt, cf. Appendix [Cl for the details. In the cases discussed below, 
the removal of a vertex using our decimation procedure preserves almost exactly the fundamental cycles and their 
affinities; the small variations observed are due to the possibile reduction of whole 3-loops to 2-loops (i.e. simple links) 
corresponding to a total loss of the affinity of the original 3-loop. The impact of this unfortunate event is difficult to 
estimate, because it depends on the topology of the graph: the removal of a vertex may lead to a crunch of a number 
of loops smaller than or equal to the degree of the removed vertex. The amount of lost affinity for each reduced loop is 
expected to be small, since it is associated with a small loop, and correspondingly small should be the loss in current 
and in entropy production. It is remarkable that the exit times of states do not affect the affinities, although they 
can affect the currents. 

Nevertheless, decimation may affect the large loops as well; a progressive and repeated removal of vertices may 
eventually reduce a large loop to a 2-loop. Unfortunately, controlling these events goes beyond our mathematical 
ability, therefore, the size of the error in the conservation of fundamental cycles under decimation remains an open 
question. 

In the figures, all black objects (vertices and links) are related to the original graph, red objects are the new ones 
formed after decimation. Solid links are part of the maximal tree, dashed links are chords. When the state labelled 
by is removed, it is linked to some other states collectively denoted as j: j'-states (linked to 0) are in number of n. 
These links are broken and all pairs of states j and j' (previously connected to 0) are connected among each other 
with a new transition rate Wj_>oWo->j' /Wq (or, if the link j — j' already exists, its transition rate is updated adding 
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that amount). We consider the simplest case where one chord at most is involved in the decimation procedure. With 
this assumption, three possibilities can be encountered: 



FIG. 10: An example of state-removal where no chords belong to the subgraph involving the removed state. Black objects 
(vertices and links) are relative to the original graph, red objects are formed by the decimation protocol. Solid links are part 
of the maximal tree, dashed links are chords. 



1. new loops v^rith no entropy production: 

The simplest case is realized when no chords connect any j to and no chords connect any j to any j', cf. Fig. llOl 
for one example. In this case, all original links j — > {A, B, C and D in Fig. [T0|) are on the spanning tree and 
no links join any j to any j' . After the links and the central state have been removed, the red links are created 
(a, b, c, d, e, f in the example): they are in a number n{n — l)/2. A number (ri^ — 3n + 2)/2 > (for any n > 2) 
are new chords (a, d and e in the example), while the remaining n — 1 are links of the new spanning tree (b, c 
and f). Therefore new loops have been created (in the example they are 3 — 4 — 2 — 3 with chord a, 3 — 1 — 2 — 3 
with chord e and 1 — 2 — 4—1 with chord d) . It is immediate to verify that the affinity of the new loops vanishes: 
for instance the loop 3 — 1 — 2 — 3 has forward transition rate given by 3 — > 0, — > 1, 1 ^> 0, — > 2, 2 — >■ 0, -> 3 
and backward transition rate given by 3 0,0 — >■ 2,2 — >• 0,0 — >■ 1,1 0,0 3 and they exactly cancel out 
(the exit rates are omitted, but they cancel out trivially): these new loops do not contribute to the entropy 
production. 



FIG. 11: An example of state-removal where a chord is removed by decimation. Black objects (vertices and links) concern the 
original graph, before decimation, red objects are the new ones formed after decimation. Solid links are part of the maximal 
tree, dashed links are chords. 

2. loop-shortening: 

Another possibility (see Fig. [TT|) is that some link j* is a chord in the original graph, which means that it 
is not in the spanning tree: e.g. link A in the example, with j* = 4. Then, state j* is connected to through 
some other unique path on the tree, possibly passing through a state j' (the unique path on the tree is also 
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represented in the figure, terminating with the hnk 3 — 0), forming a loop — j* ~ tree — j' — 0. In this case, the 
decimation of state creates the hnk j* j' (hnk "a" in the example) as a chord of the loop j* — tree — j' — j*, 
which is two steps shorter than the orginal loop. It is immediate to see that the affinity of the new loop is the 
same as that of the old one. In this case, all new links starting from j*, or from j' , and ending in another j, must 
be chords, since j* and j' are joined by a unique path on the tree which has not been touched by decimation: 
the number of new chords is larger than in the previous case, but all their loops have zero affinities. 

There is also the possibility that the loop passing through chord A is simply given by j* — — j — j* , i.e. that it 
is a 3-loop, originally belonging to the graph: this case can be put in the last category, simply exchanging the 
roles of the chords A and a: we call loop-crunching this case. 




FIG. 12: An example of state-removal where a chord connects two states which are directly linked to the removed state. Black 
objects (vertices and links) pertain to the original graph, before decimation, red objects are formed by decimation. Solid links 
are part of the maximal tree, dashed links are chords. 

3. loop-crunching: 

The last possibility is that some link j — !■ j' already existed in the original graph, which means that it is a chord 
of the loop A — j — j' — A, cf. Fig. [TH where chord a connects states 4 and 3. In this case the removal of state A 
leads us to crunch the 3-loop, making it a simple link j — j' with a new transition rate. The original loop and 
its contribution to entropy production are then lost. 

We stress that a mathematical proof of the above considerations is still lacking, although our arguments are strongly 
supported by numerical results. 

IV. CONCLUDING REMARKS AND OPEN PROBLEMS 

In this paper we support, both numerically and theoretically, the idea that fluctuations of the entropy production are 
essentially insensitive to a coarse-graining based on decimation of fast states, provided that decimation does not remove 
fundamental loops carrying net currents. The threshold of coarse-graining level which trigger the removal of such 
loops is not fully understood, but our investigation suggests that entropy production fluctuations are generally quite 
robust with respect to decimation. Moreover, this robustness does not appear directly related to the characteristic 
times of the removed states. Robustness or fragility of loops appears mostly related to the global structure of the 
network at hand. 

We applied this analysis to the network model of the biomolecule known as kinesin, discovering that no entropy 
production is lost if a coarse-graining from six to five states is performed. This observation is potentially interesting 
in biophysics, since entropy production is a fundamental property of irreversible chemical reactions, such as those 
fueling the kinesin motor protein. On the contrary, the decimation of the model from six to four states is catastrophic, 
making the model unsuitable to produce work. 

More detailed studies are necessary to quantify the entropy production variations induced by coarse graining: the 
main missing ingredient is the evaluation of the effect of decimation on the currents of the surviving loops. This will 
lead to a better understanding of the role and meaning of W* as a definition of entropy production. In particular, 
understanding the relation between Wt and macroscopic observable properties of the system may help in modelling 
non-equilibrium systems. 
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Appendix A: The decimation procedure 

In this Appendix, we summarize the coarse graining method introduced in [Tl| . Consider a master equation of the 
form of eq. ([T]). Due to the Markovian nature of the process, the time spent in a generic state n is exponentially 
distributed with average t„ = One may wish to decimate all states having an average permanence time smaller 

than a prescribed threshold At. To do that, Ref.[llJ sets to the time spent in these states. In this way, the fast 
states disappear from the description and transitions to them are redirected to other states with proper statistical 
weights. In formulae, if a state i is linked to a state j via a fast state n that must be eliminated, the transition rate 
Wi^j from i to j is renormalized to yield the rate: 

W,^j = W,^, + W,^nWn^j/W° (Al) 

If Wi-^j = 0, the decimation creates a direct connection between the surviving states, which is reminscent of the 
states that disappeared from the model under consideration. 

This procedure corresponds to an adiabatic approximation and is commutative, if the prescription of [T]| is followed. 
Once the set of states to be decimated is determined by the threshold, they can be decimated in any order without 
affecting the final result, as long as the set itself is not modified during the decimation procedure. It may happen, 
indeed, that the permanence time of some of the states selected for decimation becomes larger than At, while other 
states are decimated. The recipe of [ll| requires that this state be eventually decimated nonetheless. 



Appendix B: Effect of decimation on the current in a single loop 



In this Appendix, we investigate the effect of decimation on the current of a single loop consisting of N states. For 
convenience, let us rewrite the master equation: 



with Pq = Pn, as: 



^Pnit) = J„ - Jn-1 (Bl) 

at 



where the local current J„ is given by: 

In = P„_lW„_i^„ - PnWn^n-1. (B2) 

In a stationary state, the current is site-independent and one may write J„ — J. In particular, detailed balance and 
equilibrium hold if J = 0. The set of equation J„ = J, together with the normalization condition P„ = 1, can be 
solved for both J and the invariant measure P™'". For instance, let us proceed iteratively, as follows: 

_ Wn-l^n J 
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We obtain P„ from the last expression 

Leo - L{ ] 

= ^ ^ ^ (B4) 

A:— 1 

and by means of the normaUzation condition ^ P„ = 1, we reach the foUowing closed expression for J: 



N 

n 1 - 1 



J= ^ . (B5) 



AT AT-l ^ j 

s s w„„i^„_,_i n 

n=l j=0 ' ^ k=0 



-j + fc-l->n-j + fc 



-j + k->-n-j + k-l 



Let us now decimate one fast state, n* say, and consider the current. It is easy to show that the numerator is not 
affected by the decimation protocol defined by eq. (|Aip . Conversely, the denominator decreases by an amount AD 
which can be espressed as follows: 

AD = D^-D,= —^ +Elir^ n Z""^'^'"^'^' (B6) 

J—0 k—0 

where Dq and Dd are the denominator in (jBSP for the original and the decimated system, respectively. As AD is 
positive, the current in the decimated system is larger than in the original one, the difference being 

AJ^Jd-J^J — ^ . (B7) 
Do- AD ^ ' 

This allows us to check what happens in simple cases. For instance, eq. (|B6I) leads to AD/D = 1/N, if all state have 
same left and right jump rates (the two must be different to have a non-trivial current). If the rate of the decimated 
state is much faster than the others, eq. (jB6[) also shows that the correction decreases linearly with the separation of 
time scales, i.e. with the ratio of the average rates of the fast states and that of the other states. This is consistent 
with the picture of the current correction being essentially due to a rescaling of the times related to the elimination 
of the fast state. In other words, the magnitude of the correction seems to be always related to the ratio of the time 
spent in the fast state(s) and the time spent in the slow ones. 



Appendix C: Graphs and currents 



We consider a Markovian (continuous time) process on N states. The N states are considered as nodes of a graph. 
The transitions between different states are considered as links (edges) between nodes. 



1. Fundamental cycles 

Graph theory simplifies the classification of closed loops on a graph [l^ , identifying a set of fundamental "cycles" . 
Given a graph G with TV vertices (nodes) and E edges (links between nodes), the strategy - exemplified in Figure [Ol 
- is the following: 

• identify a maximal tree T{G), i.e. a set containing all N vertices and part of the E edges, which is connected and 
does not contain circuits. It is easy to show that T(G) has iV — 1 edges. Many maximal trees can be identified, 
but one suffices; 

• given an arbitrary maximal tree T{G), the edges of G which do not belong to T{G) are called chords of T{G), 
they are in a number i' = E — N + 1; 

• if only one chord s^, a £ [1,H' added to T(G), the new graph contains only one circuit, Cq., obtained by 
T{G) + Sa removing all edges which are not part of the circuit; therefore, from a maximal tree, u circuits can 
be generated adding the v chords; 
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FIG. 13: An example of graph with 5 states and three fundamental loops: all transitions (links) have a given orientation. A 
possible maximal tree is the one made of only solid links, with the three dashed links representing the remaining chords, which 
individuate three fundamental loops. Any other possible loop, e.g. C = l— ^2— >-4— ^5— >-l, can be decomposed in the sum 
of fundamental loops, e.g. C = Ci + C3. 



• the set of circuits obtained from the 1/ chords of a maximal graph is called a fundamental set of circuits, 
denoted by {Ci, C^} 

• orientation of edges must be introduced: each edge is assumed to be oriented in an arbitrary direction, giving 
the oriented version of G, denoted as G; then one can take a subgraph with oriented edges P, which may 
have different orientations with respect to the original orientations of the edges of G. The function Sg {P) is 
introduced for these cases: it returns 1 if the edge e is in P and has the original orientation, —1 if it is in P and 
has opposite orientation, and if e is not in P. 

• a cycle is an oriented circuit, e.g. C; a fundamental cycle is denoted by Ca- for simplicity we always choose the 
orientation of a fundamental circuit to be parallel to the orientation of its chord a, i.e. SaiCa) = 1; 

• the scalar product among cycles is defined as 

{C, = Sa{C)Sa{Cc) = Sa{C) (CI) 
where a is the chord which generates the circuit C^, this scalar product can only take three values: 0, 1 or — 1. 

• a decomposition of cycles is finally achieved: any cycle (oriented circuit) of the graph G can be linearly decom- 
posed using the fundamental set as a basis: 

(7 = ^((7,a)(7„ (C2) 



2. Currents 

The current for the co ^ to' transition is 

J{co ^ co',t) = P^{t)W^^^> - P^,{t)W^,^^. (C3) 
The stationary state value is denoted by J{co — > co'). The stationarity condition dP^^/dt = is equivalent to 

J2 J{^' w) = Vw (C4) 

which is known as Kirchhoff current law. If the transition w — >■ corresponds to the oriented edge e, its steady state 
current is also denoted as Jg. 

The ciirrent (or flux) on a fundamental circuit is defined as the steady state transtion current flowing in the chord 
a in the original direction and is denoted by Ja- For instance if a is the oriented edge corresponding to the transition 
u) u)', then Jq, = J{u) u)') and the flux of the associated cycle Ca is equal to +Ja- 

The Kirchhoff law for the steady state guarantees that a current on any edge is the sum of the currents going 
through the cycles which intersect the edge, i.e. 

V 

Je = Y,Se{CcdJcc. (C5) 
a=l 
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An edge of the graph G can be oriented in a different direction with respect to the edges of the cycles, therefore the 
sign function S,. is used. 

The fluctuating instantaneous current Jq depends on the particular realization of the Markov process; it is measured 
on a chord a as: 

+00 

So.{en)5{t-U.) (C6) 

n— — 00 

where tn is the time of the random transition e„ (an oriented edge of the graph) during a trajectory of the stochastic 
process. In brief, ja is the instantaneous and oriented rate of the transitions in the chord a, for a particular realization 
of the process. It is a stochastic variable. Its time-average (in a finite time t) is denoted by 

Ga(i) = 7 / dt'j^it'), (C7) 
which is still a stochastic variable. Some properties of Jq and Ga have been studied in [2lj |. 



3. Affinities 

The affinity of a transition a; ^> is defined as 

The affinity of a cycle C is defined as A{C) = '^^Se{C)A{e), but it can also be defined as A{C) = X^e '^e(C')5(e). 
where B{u) — > uj') = h\{Wi^^^i /W^'^^). The equivalence of these two forms is due to the fact that all Pi^(i) cancel 
out, in a cycle. For this reason the affinity of a cycle does not depend upon time, but only on the transition rates, 
which come from the "external physical constraints" , e.g. mechanical, chemical and thermodynamical forces. 

Thanks to the decomposition of cycles described above, one can linearly decompose the affinity of any cycle in 
terms of affinities of a "fundamental set of cycles" {Cq}: 

A(C) = ^(C,aM(a) (C9) 
where (., .) is the previously defined scalar product between cycles. 



4. Entropy production 

Having defined the Gibbs entropy as 

5(i) = -^P^(t)lnP„(t), (CIO) 

its time derivative can be decomposed in two parts dS/dt = deS /dt + diS/dt, where the bilinear form 

d,S 1 



dt 



^ J(a;^a;',t)A(w^a;',i) >0 (Cll) 



is considered as the internal entropy production, and the rest d^S/dt is the entropy fiux through the boundaries of 
the system of interest. In the steady state one has deS/dt = —diS/dt. 

A definition of entropy production per trajectory is given by Lebowitz and Spohn Q , see Eq. Q . It depends on a 
particular realization, i.e. it is a stochastic variable. It can also be written as: 



i5]s(e) f dt'Ut') 



^t^-yB{e) / dt'Ut'). (C12) 
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Lebowitz and Spohn have noticed that 



lim 



in the stationary state. The following relation has instead been noticed in Ref . [2ll| : 

m = Qt + Rt 

with 

a 

e^a V a J. 



(C13) 

(CM) 

(C15) 
(C16) 



The Qt term is the contribution due to the fundamental set of cycles. The "remainder" Rt has zero average (thanks 
to the Kirchhoff law Eq. (ICSp ). This implies that 



dt 



lim {Wt) = lim {Qt) = V A(d) Ja 



since 



lim(Ga(t)) = lim i / / j^{t')dt' 

t—>-00 t^OO t \Jq 



J a ■ 



(C17) 



(C18) 



Numerical comparison of the fluctuations of Qt and those of Wt show that they have identical Cramer functions (see 
Figure ini, in many examples of continuous time Markov processes. 

From eq. (jC3p . detailed balance, with respect to the invariant measure, is equivalent to 



J(w^w')=0 V(w^w'), 



(C19) 



which implies that the probability of any trajectory is equal to the probability of its time reversal. Detailed balance 
also implies that the flux on any cycle vanishes, Jq = 0, and that affinities vanish on a single edge as well as on any 
cycle, eg. A{Ca) = 0. As an immediate consequence, the internal entropy production vanishes: 



djS 
~dt 



0. 



(C20) 



Appendix D: Appendix: parameters in the kinesin model 

As sketched in Section FlI Dl the rates in the kinesin network model of Ref. l3| are adjusted to the parameters 
obtained by specific experiments. Moreover, they depend on the concentrations of the chemical species entering the 
reaction (ADP, ATP and P), as well as on the load force F. More formally, one has: 

=%/.,([A])<i>,,(^) (Dl) 

where the fc's are the experiment-specific parameters. The functions and express the dependence of the reaction 
rates on a generic chemical species X and/or on the load force F . If the transition from i to j does not involve chemical 
binding, we define lij = 1. 

Assuming diluted solutions, all reactions are diffusion-limited, so that we can assume /([AT]) ^ [X]. The $'s 
are adimcnsional functions, with the convention $(0) — 1. Theoretical considerations lead to ^ij{F) = ^ji{F) = 
2/(1 + e^'J^) for the chemical transitions, i.e. all but those between states 2 and 5. Mechanical transitions are 
parametrized by $25 — e~^^ and $52 — e^^"^^^. The x's and 9 are additional parameters obtained by experiments, 
while F — IF/kT is the adimensional force {I w 8nm being the average kinesin step length and k the Boltzmann 
constant). With these choices, the /c's are dimensionally different depending on whether they multiply a concentration 
(dimensions of rate divided by concentration, [(/iM s)~^]) or not (dimension of a rate, [s~^]). 

The parameters we used in the simulations are derived from those reproducing the results of the experiment [l9| : 
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• The values of fc's describing the experiment [l^ according to [T^ are ^25 = 3 • 10^, ^52 = 0.24, ki2 = ^45 = 2.0, 
/c2i = 100, = ftgi = = = 100, /cgs = ^32 — 0.02, fcie = fc43 = 0.02. The upper and lower cycle in 
Fig. [7] are assumed to have same parameters, apart from the transition from 5 to 4, which is determined from 
theoretical considerations as fc54 — ^21(^52/^25)^ — 6.4 10^^^. 

• Typical concentrations in the experiment are 0.5/iAf . For simplicity, we assume all of them to be kept constant 
and equal to [P] = [ADP] = [ATP] = lfiM. 

• The mechanical parameters reproducing the results of experiment are: 6 = 0.65, X12 — X45 = 0.25, 
X23 = X56 = 0.15, X34 = X61 = 0.15. In all cases, we have Xij = Xji- 

In section HlDl we considered two instances of the model. The first one is without load, F ~ 0. In this case and 
with the assumptions above, it is easy to obtain the transition rates: all the $'s and concentrations are equal to 1, so 
from Eq. (|D1|) we obtain Wi-^j = ktj: the rates are just the k's listed above. 

About the load case, the unit of the adimensional force is equal to kT/l « Q.bpN. Experiments are performed 
with forces of the order of piconewton. We took a value F = 10: substituting this value in the expression for the 
$'s leads to the following values of the transition rates, which are those used in the simulations of the model with 
load: W2^5 = 451, M^5->2 = 7.95, I^i^2 = W^4->5 = 0.3, W2^i = 15, M^s^g = M^e^i = ^^2^3 = W^^^ = 36.5, 
We^5 = W3^2 = 0.007, Wi^e = W^^s = 0.007, W^^i = 0.5 10^". 

Appendix E: List of the main symbols 

• Wi^j is the transition rates from i to j 

• is the exit rate from state i 

• Ti — l/W^ is the characteristic time of state i 

• Pn (t) is the probability of being in n at time t 

• P™^ is the invariant probability of being in n 

• At is the time threshold for decimation 

• Wi^j are the new transition rates in the decimated process 

• Wt is the Lebowitz-Spohn entropy production integrated on time t and divided by t 

• C() is the Cramer's function of the entropy production 

• /(Wt) - e~*<^(^') is the probabihty density of Wt 

• CAr(VVt) is the Cramer function in the decimated process with a time threshold At. 

• Ca is an oriented cycle of the graph 

• Ga (t) is the current on cycle a averaged on a finite time t 

• A{Ca) is the affinity associated to the oriented cycle Ca- 
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